Method and device for optically detecting the open state of a vehicle door

ABSTRACT

The invention relates to a method for detecting the position of an object within the interior or a vehicle equipped with a door ( 4 ). The inventive method is characterized by recording a sequence of images by means of a camera ( 2 ) and calculating the actual position of the object by electronically evaluating the images of the sequence of images, and generating a differential image by comparing an actual recorded image with a previously stored image. The position of the object is calculated on the basis of the differential image. To achieve this, the camera ( 2 ) records images of the door ( 4 ). The images of the door ( 4 ) are evaluated and the open state of the door ( 4 ) is established on the basis of the differential image between the closed state and the opened state.

CLAIM FOR PRIORITY

This application claims priority to PCT/DE02/02423, published in theGerman language on Jul. 3, 2002, which claims the benefit of priority toGerman application no. 10133511.3, filed in the German language on Jul.10, 2001.

TECHNICAL FIELD OF THE INVENTION

The invention relates to a device and method for detecting the positionof an object within the interior of a vehicle equipped with a door.

BACKGROUND OF THE INVENTION

WO 00 65538 discloses a differential image generated in which thecontour of the object whose position is to be detected appears only ifthe object moves. In this method, only moving objects are automaticallyrecorded in a scene, which makes it possible for images to be evaluatedsimply and reliably. A CCD or CMOS camera which is equipped with awide-angle lens and mounted as high as possible within the interior ofthe vehicle, preferably in the roof lining of the vehicle, is used forrecording images.

The camera or a different optical sensor system can be arranged betweenthe front seats; with its viewing angle it then covers the area betweenthe dashboard and the seat backrest and between the roof lining and theseat cushion.

The known method is used in order reliably to determine informationabout the position of the front passenger, in particular about theposition of the front passenger's head, within the interior of thevehicle, and to use this information for releasing an airbag system.

On the other hand, it is also known for a door contact of the vehicle tobe read out in order for this information to be evaluated for monitoringthe airbag system. A monitoring device classifies the occupancy of avehicle seat, in particular of the passenger seat, into various possibleoccupancy classes (empty seat; child-seat facing backwards; child-seatfacing forwards; adult person) and permits, by means of definedtransitional probabilities, transitions between these occupancy classes.Provided that the vehicle door is open, the airbag system should not,however, be enabled. For that reason, a door contact, for example amechanical door contact, of the vehicle generates a signal which is fedto the monitoring system. To achieve this, an electric line exists fromthe door contact to the camera or sensor system in the roof lining.

A cable has, however, the disadvantage that laying it entails a cost. Along cable brings with it additional EMC problems (EMC=electromotivecompatibility). The usually cheap door contact of the vehicle becomes asafety-relevant part because it affects the releasing of the airbag.

SUMMARY OF THE INVENTION

The invention relates to a method for detecting the position of anobject within the interior of a vehicle equipped with a door, in whichmethod a sequence of images is recorded by means of a camera and theactual position of the object calculated by electronically evaluatingthe images of the sequence of images, a differential image beinggenerated by comparing an actually recorded image with a previouslystored image and the position of the object being calculated on thebasis of the differential image.

The invention is establishes a method for reliably detecting the vehicledoor.

According to one embodiment of the invention,a camera records images ofthe door, such that the images of the door are evaluated and in that, onthe basis of the differential image between the closed state and theopened state, the open state of the door is established.

In addition to ascertaining the open state, the method can in particularalso be used for establishing from the differential image the specificopening angle of the door. This opening angle can be used in order toprevent, when the vehicle is parked in a car park, the vehicle door frombeing opened too widely, as a result of which these or the vehicle bodyparts of an adjacent vehicle or other adjacent objects could be damaged.

The method for establishing a differential image is used in anadvantageous manner in conjunction with a monitoring system. In thismethod, the camera generates, depending on the open state detected, asignal and feeds this signal to a monitoring system which, on the basisof the signal, switches on or off a security device for the protectionof vehicle occupants. By this means, the disadvantages of a door contactconnected with an electric line can be avoided.

Additionally or alternatively, the method according to the invention canbe used in order to measure the open state of a window. In this case,the camera generates, depending on the detected open state of a window,a signal and feeds this signal to a monitoring system. This can preventthe arm or head of a passenger who is leaning out of the window frombeing jammed or injured by a motorized window actuator or an object frombeing jammed or damaged by the rising window. A precondition for this,however, is that the endangered body parts of the passenger or theobject are not concealed, for example by the torso of the passenger.

When the camera detects an object within the range of the opened window,said camera supplies a signal to the monitoring system, based on whichsignal the monitoring system initializes a lockout which prevents theopened window from being closed.

The evaluation of gray-scale images proves to be a particularly suitablemethod for evaluating open states of the vehicle door or of a window.

In order, by means of a small number of gray-scale images, to reach adecision about the open state of the door and/or of a window, virtualimage areas are defined for obtaining gray-scale images, in whichvirtual image areas changes in the gray-scale values are expected ifthere is a change in the open state of the door and/or window.

In order, when evaluating gray-scale images, to have to use as littlestorage space as possible in a data memory, for each image obtained agray-scale curve is differentiated as a function of the locationaccording to the location. The result of the differentiation is storedin each case and the open state determined from comparison of the storeddata.

As an alternative to obtaining gray-scale images, three-dimensionaldistance images are recorded with a 3D camera, in order to establishfrom these the open state of the door and/or window. Measuring panelswhose mean distance to the camera is measured are used here. In afurther method, distance points are projected, after transformation intothe vehicle system, onto a plane parallel to the floor of the vehicle.The projected images are then each stored and the open state determinedfrom comparison of the stored data. Characteristic point clouds aredetermined by this means. When the door is opened, the position of theassociated point cloud changes.

The present invention also relates to a device for implementing a methodas described above.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are described in detail below withreference to the drawings, in which:

FIG. 1 shows a section through a vehicle at right angles to thedirection of travel with a plan view of the passenger seat.

FIGS. 2 a, b show a section coplanar with the vehicle floor with a planview of the passenger seat when the door is closed and open.

FIGS. 3 a, b show a plan view of the passenger door from inside withedge boxes in a closed and in an opened state.

FIG. 3 c shows an edge box which covers an edge area of the window andof the inside of the door.

FIG. 3 d shows a curve of gray-scale values plotted over the edge boxfrom the window area to the area of the inside of the door.

FIG. 3 e shows the first derivation of the curve of gray-scale valuesaccording to location.

FIGS. 4 a, b show a lateral plan view of a vehicle passenger and thepassenger door with measuring panels for a 3D camera and/or a sketchwith projections of measuring points.

FIGS. 5 a, b show image dots obtained with the 3D camera aftertransformation onto a plane coplanar with the vehicle floor.

DETAILED DESCRIPTION OF THE INVENTION

A CCD or CMOS camera 2 which is equipped with a wide-angle lens 9 ismounted in a vehicle with a roof lining 1 (FIGS. 1; 2 a, b). The camera2 is preferably aligned in the center of the vehicle or alternatively inthe area of the upper steering spar of the vehicle body such that theimage area of the camera above a seat 3 covers a door 4 both in the areaof the inside of the door 5 with a door handle 6 and a window 7. A lineA denotes the distance between the camera 2 and the door 4 in a closedstate at a first point and a line B the distance at a second point. Thelines A, B correspond to the lines A′, B′ in an open state.

When gray-scale images are recorded with the camera 2, edge boxes 10 to16 (FIGS. 3 a, b) are placed around the frame of the window 7. The edgeboxes 10 to 16 are virtual image areas which are placed on prominentpoints of the door 4 having large changes in gray-scale values.

The edge box 10 which covers the lower area of the window 7 and an areaof the inside 5 of the door is M pixels long in the y-direction, i.e. inthe working direction of the camera 2 and N pixels wide in thex-direction. Inside the edge box, the coloration of a door seal 8, whichis black, can also be detected as well as the gray-scale values for theinside 5 of the door, which is for example gray, and the section of thewindow 7, which is light.

Thus, for the three areas inside the edge box 10 which are scannedpixel-by-pixel by the camera 2, a representation is produced of thegray-scale values which are shown as a function in the working direction(y-direction in FIG. 3 c) in FIG. 3 d over the pixels M. Here, thecourse of the gray-scale values of the window 7 (transparent), door seal8 (black) and inside 5 of the door (gray) can be detected.

FIG. 3 e shows the derivative function of the gray-scale value functionfrom FIG. 3 d according to location, i.e. according to working direction(y-direction in FIG. 3 c, x-direction in FIG. 3 d). This course isobtained in practice with the aid of a cut-off filter. The positions ofthe edges between the window 7 and the door seal 8 on the one hand(position I) and between the door seal 8 and the inside 5 of the door(position II) on the other are expressed here. The locations of theedges (positions I and II) are typical for this door position, which isthus uniquely characterized.

With this method of representation, very little storage space isrequired since for each of the edge boxes 10 to 16 only the position oftwo diagonally opposite corners, the working direction of the camera 2and a threshold value for the gray-scale values have to be adjusted. Adifferent threshold value can be individually selected for each edge box10 to 16.

By comparing the positions of the edge boxes 10 to 16 with the door 4closed (FIG. 3 a) and with the door 4 open (FIG. 3 b), the change ofperspective produces a shift of the edge boxes 10 to 16 relative to thedoor 4. The edges of the edge boxes 10 to 16 change even when there arevery slight changes in the opening angle of the door 4. Consequently,the angle at which the door 4 is open can also be ascertained. Since theconcealment of individual edge boxes 10 to 16 by occupants of thevehicle is in principle possible, a certain minimum number of edge boxes10 to 16 is preferably specified, in order that, even if some of themare concealed, reliable measurement results can be obtained from theother unconcealed edge boxes, in which the edge positions match thestored edge positions.

In another embodiment, the camera 2 is fashioned as a 3D camera. In thiscase, measuring panels 17 to 19 (FIG. 4 a) are defined which map partsof the door 4 and inside which the mean distance to the camera 2 ismeasured. By comparing the actual distance relative to a stored distancewhich was measured for example for the closed door 4, it can beconcluded whether the door 4 is open or closed. The measuring panels 17to 19 are projected onto a projection plane, e.g. the vehicle floor(birdview). From the relative change in the mean distances of themeasuring panels 17 to 19 from one another when the door is closed (FIG.4 b) (position I) and when the door 4 is open (position II), the openingangle of the door 4 is determined.

In a further method (FIGS. 5 a, b), the measuring points are projectedonto a plane coplanar with the vehicle floor. This produces along ay-axis (axis at right angles to the direction of travel of the vehicle)a characteristic grouping of points whose position along thelongitudinal axis of the respective point cloud reproduces the positionsof objects. Thus, one grouping of points reproduces the position of thedoor 4 in a closed state (FIG. 5 a), while positions 20, 21 representthe legs of a passenger and position 22 the body and the backrest of theseat 3.

In an open state (FIG. 5 b), the point cloud which represents the door 4has changed its position, while all other point clouds 20 to 22essentially have the same position. The direction of movement of thepoint cloud for the door 4 is then searched for, either by means of adoor model including the known position of the pivot of the door 4 orwithout a model by drawing on transformations known from the literature,e.g. the Hough transformation (cf. K. R. Castleman, Prentice Hall; P.Haberäcker, Carl Hausen Verlag publishers, Munich, Vienna; B. Jähne, H.Hauβecker, Computer Vision and Applications, Academic Press.

1. A method for detecting a position of an object in a vehicle and/or anobject forming a part of the vehicle, comprising: recording a sequenceof images of a door of a vehicle with a camera; generating adifferential image by comparing a currently recorded image of the doorwith a previously stored image of the door; determining whether the dooris in an open state or a closed state by evaluating the differentialimage; and if the door is in the open state, switching off a securitydevice protecting at least one occupant of the vehicle.
 2. The methodaccording to claim 1, which further comprises determining an openingangle of the door by evaluating the differential image.
 3. The methodaccording to claim 1, which further comprises: determining whether awindow located in the door is open by evaluating the differential image;and if an object is within an open area of a frame of the door,generating a signal indicating the presence of the object within theopen area; the open area of the frame of the door being formed when thewindow is open.
 4. The method according to claim 3, which furthercomprises preventing the window from being closed if the signalindicates the presence of the object within the open area.
 5. The methodaccording to claim 1, which further comprises using gray-scale imageswhen performing the step of determining whether the door is in the openstate or the closed state.
 6. The method according to claim 4, whichfurther comprises defining virtual image areas to obtain the gray-scaleimages such that changes in a plurality of gray-scale values occur ifthere is a change in the open state of the door.
 7. The method accordingto claim 6, which further comprises, for each of the gray-scale images,defining a curve of gray-scale values as a function of a location,obtaining a differentiated curve by differentiating the curve ofgray-scale values with respect to the location, storing thedifferentiated curve, and using the differentiated curve when performingthe step of determining whether the door is in the open state or theclosed state.
 8. The method according to claim 1, which furthercomprises: obtaining three-dimensional distance images with the camera;and using the three-dimensional distance images when performing the stepof determining whether the door is in the open state or the closedstate.
 9. The method according to claim 8, which further comprises:after transforming distance points into a vehicle system, projecting allof the distance points onto a vehicle floor or onto a plane parallel tothe vehicle floor; storing the projected images; and using the storedprojected images in a comparison when performing the step of determiningwhether the door is in the open state or the closed state.
 10. Themethod according to claim 8, which further comprises defining pointclouds of the door.
 11. The method according to claim 8, wherein thesecurity device is an airbag system.
 12. A method for detecting aposition of an object in a vehicle and/or an object forming a part ofthe vehicle, comprising: recording a sequence of images of at least aportion of a door of a vehicle with a camera; generating a differentialimage by comparing a currently recorded image of at least the portion ofthe door with a previously stored image of at least the portion of thedoor; determining whether a window located in the door is in an openstate by evaluating the differential image; and if an object is withinan open area of a frame of the door, generating a signal indicating thepresence of the object within the open area; the open area of the frameof the door being formed when the window is open.
 13. The methodaccording to claim 12, which further comprises preventing the windowfrom being closed if the signal indicates the presence of the objectwithin the open area.
 14. The method according to claim 12, whichfurther comprises using gray-scale images when performing the step ofdetermining whether the window is in the open state.
 15. The methodaccording to claim 14, which further comprises defining virtual imageareas to obtain the gray-scale images such that changes in a pluralityof gray-scale values occur if there is a change in the open state of thedoor.
 16. The method according to claim 15, which further comprises, foreach of the gray-scale images, defining a curve of gray-scale values asa function of a location, obtaining a differentiated curve bydifferentiating the curve of gray-scale values with respect to thelocation, storing the differentiated curve, and using the differentiatedcurve when performing the step of determining whether the window is inthe open state.
 17. The method according to claim 12, which furthercomprises: obtaining three-dimensional distance images with the camera;and using the three-dimensional distance images when performing the stepof determining whether the window is in the open state.
 18. The methodaccording to claim 17, which further comprises: after transformingdistance points into a vehicle system, projecting all of the distancepoints onto a vehicle floor or onto a plane parallel to the vehiclefloor; storing the projected images; and using the stored projectedimages in a comparison when performing the step of determining whetherthe window is in the open state.
 19. The method according to claim 18,which further comprises defining point clouds of the door.
 20. A devicefor detecting a position of an object in a vehicle and/or an objectforming a part of the vehicle, the device comprising: a camera recordinga sequence of images of a door of a vehicle; and a monitoring deviceconfigured to: generate a differential image by comparing a currentlyrecorded image of the door with a previously stored image of the door;determine whether the door is in an open state or a closed state byevaluating the differential image; and switch off a security deviceprotecting at least one occupant of the vehicle if the door is in theopen state.
 21. The device according to claim 20, in combination withthe security device, wherein the security device is an airbag system.22. A device for detecting a position of an object in a vehicle and/oran object forming a part of the vehicle, the device comprising: a camerarecording a sequence of images of at least a portion of a door of avehicle; and a monitoring device configured to: generate a differentialimage by comparing a currently recorded image of at least the portion ofthe door with a previously stored image of at least the portion of thedoor; determine whether a window located in the door is in an open stateby evaluating the differential image; and generate a signal indicatingwhether an object is within an open area of a frame of the door; theopen area of the frame of the door being formed when the window is open.23. The device according to claim 22, wherein the signal generated bysaid monitoring device prevents the window from being closed if thesignal indicates that the object is within the open area.